productionizing nlp model
Productionizing NLP Models
Lately, I have been consolidating my experiences of working in different ML projects. I will tell this story from the lens of my recent NLP project to classify phrases into categories -- A multiclass single label problem. Making AI teams is quite tricky. If you don't have the skillsets inside your company, you have to plan hiring. Since every project has a start and end time, it's difficult to have the entire team from the start.
Productionizing NLP Models
Lately, I have been consolidating my experiences of working in different ML projects. I will tell this story from the lens of my recent NLP project to classify phrases into categories -- A multiclass single label problem. Making AI teams is quite tricky. If you don't have the skillsets inside your company, you have to plan hiring. Since every project has a start and end time, it's difficult to have the entire team from the start.
Productionizing NLP Models
It's just super important to fix up the project structure in the beginning for the code to evolve in a structured way. We took considerable time and did many discussions before we converged. Have a look at this to start with a basic scaffold. This is how we train models on EC2/local and backup code, data, models and reports on s3. The directory structure is created automatically by the preparation and train class.